Intel® Open Source HD Graphics and Intel Iris™ Plus Graphics

Programmer's Reference Manual

For the 2016 - 2017 Intel Core™ Processors, Celeron™ Processors,

and Pentium™ Processors based on the "Kaby Lake" Platform

Volume 10: HEVC Codec Pipeline (HCP)

January 2017, Revision 1.0

HEVC

ii

Doc Ref # IHD-OS-KBL-Vol 10-1.17

Creative Commons License

You are free to Share - to copy, distribute, display, and perform the work under the following

conditions:

 Attribution. You must attribute the work in the manner specified by the author or licensor (but

not in any way that suggests that they endorse you or your use of the work).

 No Derivative Works. You may not alter, transform, or build upon this work.

Notices and Disclaimers

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO

LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS

IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE

FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY

EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING

LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR

INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

A "Mission Critical Application" is any application in which failure of the Intel Product could result,

directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS

FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS

SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES

OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE

ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY,

PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION,

WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE,

OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers

must not rely on the absence or characteristics of any features or instructions marked "reserved" or

"undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for

conflicts or incompatibilities arising from future changes to them. The information here is subject to

change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which

may cause the product to deviate from published specifications. Current characterized errata are

available on request.

Implementations of the I2C bus/protocol may require licenses from various entities, including Philips

Electronics N.V. and North American Philips Corporation.

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and other countries.

* Other names and brands may be claimed as the property of others.

HEVC

Doc Ref # IHD-OS-KBL-Vol 10-1.17

iii

Table of Contents

High Efficiency Video Coding (HEVC) Introduction .................................................................. 1

Scope ............................................................................................................................................................................ 1

Summary of Features ..................................................................................................................... 1

HCP Hardware Pipeline Features ....................................................................................................................... 1

HEVC Decoder Features......................................................................................................................................... 2

HEVC Encoder Features ......................................................................................................................................... 2

Architecture Overview ................................................................................................................... 5

VP9 Decoder .............................................................................................................................................................. 5

HCP Command Summary .............................................................................................................. 5

Workload Command Model ................................................................................................................................ 5

HCP Command Sequence Examples............................................... Error! Bookmark not defined.

HCP Decoder Command Sequence ......................................................................................................... 7

HCP Encoder Command Sequence ......................................................................................................... 8

Memory Address Attributes ............................................................................................................................... 10

HCP Pipe Common Commands ....................................................................................................................... 10

Buffer Size Requirements .................................................................................................................................... 11

VP9 Common Commands .................................................................................................................................. 13

HCP Common Commands.................................................................................................................................. 13

VP9 Commands ...................................................................................................................................................... 14

Tile Size and CU Stream-out Records ....................................................................................................... 14

Stream-in Probability Table ............................................................................................................................. 1

Stream-in formats for creating compressed header ............................................................................. 1

Probability Statistics Counters Table ........................................................................................................... 2

VP9 PAK Quant Lookup Tables ...................................................................................................................... 1

SB, CU/PU and TU Sizes – Encoder Only .................................................................................................. 15

Definition of the VP9 CU Record Structure - Encoder Only ............................................................. 16

Allowed SB Size Encoder Only ..................................................................................................................... 21

HCP Commands ...................................................................................................................................................... 21

Multipass flow during BRC and SAO ......................................................................................................... 22

CU and Slice level stat streamOut............................................................................................................... 23

Definition of the CU Record Structure for Ext Interface – Encoder Only..................................... 24

HEVC

iv

Doc Ref # IHD-OS-KBL-Vol 10-1.17

LCU, CU, TU, and PU Sizes – Encoder Only ............................................................................................. 28

Allowed LCU Size – Encoder Only ............................................................................................................... 32

HEVC Error Concealment ............................................................................................................. 33

VP9 Register Definitions .............................................................................................................. 35

Register Attributes Description ........................................................................................................................ 35

VP9 Register Map ................................................................................................................................................... 35

VP9 Encoder Register Descriptions ................................................................................................................. 35

HEVC Register Definitions ........................................................................................................... 35

Register Attributes Description ........................................................................................................................ 36

HCP Decoder Register Map ............................................................................................................................... 36

HCP Decoder Register Descriptions ............................................................................................................... 36

HCP Encoder Register Map ................................................................................................................................ 36

HCP Encoder Register Descriptions ................................................................................................................ 36

HUC Status Register Descriptions ................................................................................................................... 37

Acronyms and Applicable Standards ......................................................................................... 37

Acronyms and Abbreviations ............................................................................................................................ 37

HEVC

Doc Ref # IHD-OS-KBL-Vol 10-1.17

1

High Efficiency Video Coding (HEVC) Introduction

The HEVC Codec Pipeline (HCP) is a fixed function hardware video codec responsible for decoding and

encoding HEVC (High Efficiency Video Coding) video streams.

Scope

The primary scope of the HCP BSpec document is to provide a description of the HCP commands

processed by the Video Command Streamer (VCS). The secondary scope is to provide a description of

the status registers on the Message Channel Interface to support encoding and decoding of the HEVC

video format.

The BSpec sections include:

 Summary of Features

 Architecture Overview

 Commands

 Register Definitions

 Acronyms and Applicable Standards

Summary of Features

The following sections define the HEVC Decoder and Encoder general features and the features specific

to HEVC decoding and encoding, respectively.

HCP Hardware Pipeline Features

 Supports both decoder and encoder functions, setup on a per picture basis:

 Hardware acceleration provides Ctb/CU level decode and encode.

 No context switch is supported within a frame process.

 Supports Video Command Streamer (VCS):

 Shared with MFX HW pipeline, and at any one time, only one pipeline (MFX or HCP) and one

operation (decoding or encoding) can be active.

 Supports Message Channel Interface:

Feature

Supports Tile-YS and Tile-YF.

Supports Tile-Y Legacy.

 Supports NV12 video buffer plane:

 Supports 4:2:0, 8-bit per pixel component (Y, Cb and Cr) video.

 Supports 8Kx8K frame size.

HEVC

2

Doc Ref # IHD-OS-KBL-Vol 10-1.17

HEVC Decoder Features

 Supports full-featured HEVC Main Profile standard, up to Level 6.2.

 Supports the long format HW decoding interface:

 All headers (SPS, PPS, Slice Header) are parsed and decoded outside the HCP HW pipeline.

They are then fed to the HW through a set of HCP state commands.

 Supports inner-loop decode with hardware entry points for Encoder.

 Error detection/resiliency down to the Ctb/CU level.

HEVC Encoder Features

 Supports ENC-PAK architecture

 Supports multiple pass BRC rate control operation flow

 Supports the HEVC Main Profile standard, with certain restrictions on the feature set and coding

parameters, listed in the following table:

HEVC Encoder Features and Restrictions

Note that there is a difference between what PAK supported and what ENC supported. A

feature/function that is supported in the PAK, does not necessary being supported by ENC and

MediaSDK and the like.

Coding Tool

Gen9

Support

Gen9

Restriction

Comments

LCU Size

Yes

(spec)

Support all 3 sizes: 16x16, 32x32, 64x64.

CU Size

Yes

(spec)

Support 8x8, 16x16, 32x32, 64x64.

Max 64 CU per LCU; min. CU size intel supported is 8x8 for all LCU size.

PU Partition

Yes

(spec)

Support all inter symmetric (square) and asymmetric (non-square) PU

partitioning, according to HEVC spec. PU Size for inter : Smallest allowed

is 4x8 and 8x4, and they cannot be bidirectional. Inter 4x4 PU is not

allowed in Main Profile.

TU QuadTree

Partial (intel)

max depth is

set to 3

Max depth is 3 (64x64 CU with 4x4 TU). Decoder supports this depth,

but probably no need to search this for encode. Better to just split CU.

HM common conditions set the max to 2 for both inter and intra.

Intel Encoder only supports 2 levels of quad-tree. That is,

max_transform_hierarchy_depth_inter/intra <= 2.

Max num of TUs per CU is 16.

AMP

Yes

(spec)

Asymmetric Motion Partition (rectangular PU partitioning - 2NxnU,

2NxnD, nLx2N or nRx2N). Available only for 64x64 to 16x16 CU.

AMVP

Yes

(spec)

Adaptive/Advanced Motion Vector Prediction : spatial and PU-based

temporal co-located MV candidates with scaling. Logic available from

decoder. HW PAK is supporting temporal MV candidates.

HEVC

Doc Ref # IHD-OS-KBL-Vol 10-1.17

3

Coding Tool

Gen9

Support

Gen9

Restriction

Comments

Merge

Yes

(spec)

Merge Skip and Regular Merge. Max. 5 MV Merge candidates (4 spatial

+ 1 temporal co-located) with scaling. Logic available from decoder.

[merge_flag, merge_index, skip_flag]

Parallel Motion

Merge

No (intel)

Tool for parallel decode of MVs. Since this isn’t constrained by Main

Profile, the decoder has to meet performance targets in the worst case

anyway.

MC

Interpolation

Filter

Yes

(spec)

¼-pel Luma MV precision, 1/8-pel Chroma MV precision. 8-tap Luma

filtering for both ½-pel and ¼-pel locations (1-pass). 4-tap Chroma

filtering. Use separable (first horizontal then vertical 1-D filtering) filter

coefficients. Not all filter kernels are symmetrical and can map into

simple arithmetic. It is a DCT-IF based filter. All operations are within 16-

bit data.

Weighted

Prediction

Yes

(spec)

Free for PAK since decoder already has it. Cross fade and fade-in/out

detection required by VME. VME must supply weights, implicit weighted

prediction is not supported in HEVC. Only explicit weight is supported.

They are both unidirectional and bidirectional.

Combined

Reference

Frame List

No (intel)

Combine List0 and List1 into a single list to remove uni-prediction

signaling overhead.

Intra modes

Yes

(spec)

33 directions and DC/Planar modes, with adaptive pre-filtering on

reference pixels and boundary smoothing.

Intel ENC-VME Mode search reduction for large CUs (64x64 and 32x32)

for performance speed up with minimal loss.

IPCM (intra)

No (intel)

Can be disabled completely by SPS, or only allowed at certain CU sizes.

No bit maximum on CUs in any profile/level (yet), so as of today there’s

no mandate to support this for encode.

Gen9 PAK does not support IPCM, so ENC cannot generate IPCM mode.

Constrained

Intra

No (intel)

Allow only intra neighboring blocks for current block intra-prediction.

Enabling this is a coding loss and does not result in a performance

improvement in HW designs.

2D DCT

Transform

Yes

(spec)

Square shape only; 32x32, 16x16, 8x8 and 4x4.

Transform Skip

Evaluation

No (intel)

ENC will

estimate the

use of

transform skip

Significant coding gains for screen content (PowerPoint etc.). This tool is

disabled in the common conditions but isn’t explicitly disallowed by

Main Profile. FQ is not bypass.

Sign bit hiding

No (intel)

Coding gain by removing one bypass bin per TU. Requires some smarts

in the PAK.

Trellis

No (intel)

Trellis Quantization

SAO

No (intel)

Difficult to implement in single pass, performance impact in 2-pass or

with previous frame search. Needs investigation. Decoder will support it.

HEVC

4

Doc Ref # IHD-OS-KBL-Vol 10-1.17

Coding Tool

Gen9

Support

Gen9

Restriction

Comments

Loop Filter

across

tiles/slices

boundary

No (intel)

Can be disabled for tiles and or slices in SPS, so that filter across all tiles

and slices boundaries. Main profile doesn’t constrain.

Scaling List

Yes

(spec)

This uses the default (or custom) qp adjustment on a per-frequency

basis within a TU. Good coding improvement over flat scaling.

dQP

Partial (intel)

Yes for LCU;

No for CU

Being able to change QP per LCU or even up to once per 8x8 CU can

lead to significant coding gains. Not sure how easy it is for VME to

decide qp values though.

Chroma QP

offset

No (intel)

No ROI.

Dependent

slices

No (intel)

It is now part of Main Profile, but Intel will not support it. SW can

perform the slice repackaging without re-encoding.

Tiles

No (intel)

Although in Main Profile, it results in coding loss and doesn’t improve

performance on HW. SW parallel processing (multithreaded) tool

Wavefront (aka

WPP)

No (intel)

Latest Main Profile spec has included Wavefront. We got a feedback

that this feature is highly desirable to support high performance

multithreading HEVC decoder.

Lossless

coding

No (intel)

Note: this is not the same as IPCM. Also details of this are in flux.

Interlaced

Video

No (intel)

Only progressive video encoding is supported.

LM mode

No

(spec)

Chroma-from-luma intra prediction (Linear Mode) is not allowed in

Main Profile (yet).

NSQT

No

(spec)

Non square transform is not allowed in the Main Profile

ALF

No

(spec)

Expensive and not in Main Profile currently. If decoder is going to

support it, may consider for encoder as well.

Entropy slices

No

(spec)

Not allowed in Main Profile, it is a SW parallel processing

(multithreaded) tool.

Slice

granularity !=

0

No

(spec)

Not allowed by Main Profile, and highly likely to be removed from the

standard completely.

HEVC

Doc Ref # IHD-OS-KBL-Vol 10-1.17

5

Architecture Overview

This section discusses HEVC features as follows:

Name

VP9 Decoder

The HCP Decoder, following diagram, is a hardware based pipeline that accelerates the stream decode

for an external Host Processor. The Host Processor is defined as a system processor external to the HCP

that is responsible for high level parsing of the elementary stream and workload creation.

HCP Command Summary

The HCP is configured for encoding or decoding through a set of batch commands defined in the

following sections. The software driver builds a frame level workload using these commands and stores

these workloads in graphics memory where they are fetched by the Video Command Streamer (VCS) and

presented to the HCP for processing. The commands are processed by the Workload Parser within the

HCP and the hardware is configured by the Workload Parser prior to each frame level encode or decode.

A workload is defined as a set of commands necessary to encode or decode one frame.

The software driver is required to read the HCP disable fuse to determine if the HCP is enabled. If it is

disabled, then the software driver must not enable HCP batch commands to be sent to the HCP or a

hang event may occur. Only when the HCP is enabled through the fuse, should the batch commands be

sent to the HCP.

Workload Command Model

DWord0 of each command is defined in HCP DWord0 Command Definition. The HCP is selected with the

Media Instruction Opcode “7h” for all HCP Commands.

HCP DWord0 Command Definition

DWord

Bits

Description

0